This invention relates to analog-to-digital conversion and, in particular, to a method and apparatus for converting an analog signal of limited dynamic range to a large dynamic range, floating-point, digital representation.
Integrating sensors are used in many applications to measure physical quantities. For example, photodiodes are often used as light sensors to measure the intensity of incident light. In such an application, the photodiode is configured to accumulate (integrate) charge that arises from electron-hole pair formation at the junction of the photodiode as light strikes the junction. The amount of charge that accumulates over a period of time is proportional to the intensity of the light impinging on the junction and, therefore, the accumulated charge can be used to quantify the intensity of the light.
For illustrative purposes, this application will only discuss integrating sensor systems used to measure light intensity. However, the methods and apparatus of both the prior art and the present invention apply equally well to integrating sensor systems in general, including those that are used to measure other physical quantities.
A simplified, typical, prior art integrating sensor system 10 is shown in FIG. 1. The system comprises a sensor 12, a reset switch 14, a sampling switch 16, a capacitor 18, an analog processing circuit 20 (such as an amplifier, buffer, or a simple conductor), and an analog-to-digital converter (ADC) 22. Sensor 12 provides an analog signal in response to an applied stimulus. The analog signal from sensor 12 is applied to a first terminal of reset switch 14 and to a first terminal of sampling switch 16. Reset switch 14 has a second terminal which connects to a reference voltage (e.g. the positive power supply). Sampling switch 16 has a second terminal which connects to a first terminal of capacitor 18 and also connects to an input terminal of analog processing circuit 20. Capacitor 18 has a second terminal connected to a reference voltage (e.g. ground), and analog processing circuit 20 has an output terminal connected to an input terminal of ADC 22. ADC 22 has an output terminal for producing a digital output that corresponds to the analog signal at its input terminal. Although the system of FIG. 1 could be used in other applications to measure other physical quantities, let us assume that sensor 12 is a photodetector and that sensor system 10 of FIG. 1. is used to measure the intensity of light that is incident on sensor 12. Photodetecting sensor 12 could be a photodiode or any other device that produces electrical charge in response to incident light. The rate of charge production (i.e. the photo-current produced by the photodetector) is proportional to the intensity of the incident light that strikes the sensor.
The system of FIG. 1 operates as follows: To commence the charge integration process, reset switch 14 closes momentarily to reset the sensor to a known state (e.g. zero accumulated charge or to a reference level such as the positive power supply). Reset switch 14 then opens to allow sensor 12 to begin integrating (i.e. collecting charge). After an integration time T, sampling switch 16 closes, thereby allowing the accumulated charge to pass onto charge collecting capacitor 18 and produce a corresponding analog voltage V(T) at the input terminal of analog processing circuit 20. This voltage V(T) is proportional to the amount of charge that accumulated in sensor 12 over the integration time T. Analog processing circuit 20 then amplifies, buffers, or otherwise processes this analog voltage to produce a corresponding analog signal S(T) at the input of ADC 22. ADC 22 then converts this analog signal to a digital representation (typically a binary number) at its output.
Although this approach works reasonably well, the range of light intensities that can be measured is limited by the relatively small dynamic range of the system 10. The relationship between the light intensity and the charge collected Q(T) can be described by Q(T)=IT, where I is the photo-current from sensor 12. I is proportional to the intensity of the incident light. As explained above, this charge Q(T) is dumped onto capacitor 18 at the end of the integration cycle to produce a voltage V(T) according to the equation V(T)=Q(T)/C, and this voltage V(T) is then converted by analog processing circuit 20 to a corresponding signal S(T).
Dynamic range is specified as the ratio of the maximum signal swing that analog processing circuit 20 can produce to the minimum signal level from analog processing circuit 20 that can be meaningfully detected. Thus, to quantify the dynamic range of sensor system 10, let Qmax denote the maximum charge capacity of sensor 12 and let Ss denote the maximum signal swing from analog processing circuit 20 (because Ss is determined either by the sensor""s charge collecting capacity Qmax or by saturating voltage swings in analog processing circuitry 20, Ss does not necessarily correspond to Qmax). Let Qn denote the minimum charge noise signal from sensor 12, and let Sn be the minimum noise signal from analog processing circuit 20. Sn is due to the sum of dark signal noise, thermal noise, and other noise sources such as shot noise in the MOSFET transistors of analog processing circuit 20. Finally, let Smin (corresponding to a charge Qmin) denote the minimum signal which can be meaningfully detected. Smin=xcex1Sn where xcex1 is typically a small number (i.e. 1-4). Thus, the dynamic range of system 10 is specified by Ss/Smin.
For a given integration time T, since the maximum non-saturating photo-current that can be detected is an Imax corresponding to Ss and the minimum photo-current that can be detected is an Imin corresponding to Smin, the dynamic range of the system can also be specified by Imax/Imin=Ss/Smin. If the light intensity is too high, the sensor system saturates at Ss. The sensor system cannot differentiate any incident light that has a higher intensity than that of the critical light intensity that exactly produces Imax and Ss. Conversely, if the light intensity is too low, the sensor""s signal is dominated by noise and is effectively lost in Sn (i.e. S(T)xe2x89xa6Smin). Although varying the integration time varies both Imax and Imin (since Q(T)=∫I dt), the dynamic range remains unchanged. In other words, although varying the integration time T will vary the absolute maximum and minimum light intensities that can be measured, the relative light intensity resolution of the system remains unchanged at Ss/Smin.
This limited dynamic range can seriously hamper the performance of a sensor system. For example, consider a two-dimensional array of photo-sensors (i.e. pixels) used to capture an image for a digital camera. The luminance of typical natural and man-made scenes often span 5 orders of magnitude. However, the dynamic range of typical light sensors, such as photodiodes, is much smaller. Consequently, when a scene contains a wide range of light intensities such as bright sunlight and dark shade, the resulting image obtained by the photo-sensor array loses many pictorial details. If the integration time T is minimized such that the brightest portions of the scene do not saturate their corresponding photo-sensors, many pictorial details of the darker areas will be lost. These details will be lost because the small signals produced by the photo-sensors corresponding to the darker areas of the scene will not be sufficient to overcome the noise signal Sn in those photo-sensors. Conversely, if the integration time T is maximized such that details of the darkest portions of the scene are discernible by their corresponding photo-sensors, the brighter areas of the scene will saturate their corresponding photo-sensors, and the pictorial details of the brighter portions of the scene will be lost. In this manner, normal electronic image acquisition is often characterized by images with saturated or cutoff pixels, or both.
The general problem can be better understood with reference to FIG. 2. which illustrates how an ideal, noiseless, monotonic analog signal Sxe2x80x2(t) from an integrating sensor is corrupted and limited by a generic sensor system 30. In FIG. 2, generic sensor system 30 is represented by a summing circuit 32, a signal limiter 34, and a signal acquisition circuit (e.g. ADC) 36. Summing circuit 32 and signal limiter 34 are not meant to represent physical blocks in a real sensor system, but instead they are shown to represent how various non-ideal elements within a generic sensor system effectively limit the resolvable dynamic range of the ideal analog signal Sxe2x80x2(t). In FIG. 2, the ideal analog signal is first corrupted by a noise signal Sn which additively combines with the signal Sxe2x80x2(t) at the summing circuit 32. This additive noise signal limits the smallest resolvable signal S(t) to Smin since the noise-corrupted signal must be larger than the added noise Sn to retain meaningful accuracy (Smin must be xcex1 times larger than Sn, according to our earlier definition). The noise-corrupted analog signal then passes through signal limiter 34 which limits the maximum level of S(t) to Ss if the analog signal is monotonically increasing or to xe2x88x92Ss if the analog signal is monotonically decreasing. In other words, signal limiter 34 limits the magnitude of the analog signal to Ss. In this manner, generic sensor system 30 limits the dynamic range of the final sensor signal S(t) to Ss/Smin. However, we want to be able to resolve an analog signal Sxe2x80x2(t) whose value at time T would be larger than Ss if the signal limiter 34 were not present in system 30 while still being able to resolve an analog signal Sxe2x80x2(t) as small as Smin. Prior art techniques do not provide an effective solution for this problem.
Psycho-visual theory (Weber""s Law) tells us that human vision is less sensitive to intensity variations in a strong light stimulus than in a weak light stimulus. In other words, as the overall light intensity of a scene increases, the human eye becomes less sensitive to small variations in light intensity. For example, using arbitrary units, in a scene with 100 luminosity units, the eye may be able to discern intensity variations as small as 1 unit. However, in another scene with 1000 luminosity units, the eye will only be able to discern intensity variations of 10 units. Thus, although the human eye has a huge overall dynamic range of about 12 orders of magnitude (humans can see in almost absolute darkness in one extreme to directly into bright, on-coming headlights in the other), the human eye""s resolution within any particular lighting condition is much smaller.
Weber""s Law corresponds well to floating-point representation. To illustrate this correspondence, consider a commonly used form of decimal floating-point representation known as scientific notation. Scientific notation represents any number by several digits that comprise a mantissa and several other digits that comprise to an exponent. The mantissa holds all the significant digits of the number being represented, whereas the exponent indicates the magnitude of the number being represented. For example, using scientific notation, the number 10,600 would be represented by 1.06xc3x97104. The 3-digit mantissa 1.06 indicates that the number is only significant (accurate) to 3 digits including the leftmost digit. The exponent 4 indicates the order of magnitude of the number, and in this case, it indicates that the mantissa should be multiplied by 104 to recover the entire number. This use of scientific notation indicates that the number is accurate only to +/xe2x88x9250 units since the mantissa holds only 3 digits, the lowest of which is the 100""s position. In a similar manner, the number 106,000 would be expressed in scientific notation as 1.06xc3x97105. As before, the same 3-digit mantissa 1.06 indicates that the number is only significant (accurate) to 3 digits including the leftmost digit. However, although the mantissa is exactly the same as before, since the exponent has changed from 4 to 5, this indicates that the second number is one order of magnitude greater than the first number. This also indicates that the second number is significant only to +/xe2x88x92500 units. This is because although the mantissa continues to hold 3 digits, the least significant one is now in the 1000""s position. Weber""s Law is similar in that the human eye""s relative resolution within any particular lighting condition is constant (i.e. the same number of mantissa digits) despite the fact that the eye can see over a huge range of lighting conditions (i.e. orders of magnitude in overall light intensity of a scene).
To attempt to mimic Weber""s Law, many imaging systems favor non-linear quantizing to efficiently encode the visual information of the scene. Often, a high-resolution ADC followed by digital conversion logic is used to perform non-linear quantizing, but this is an expensive approach that requires an expensive ADC and throws away most of its resolution. For example, a 10-bit ADC may be needed to produce a 6-bit non-linear code. Additionally, non-linear quantizing does not increase a sensor system""s fundamental dynamic range.
There are numerous prior art attempts at increasing a sensor system""s effective dynamic range. Most of these approaches can be divided into three categories: integration time control, gain control, and compression of the response curve. Some of these approaches will now be discussed as applied to a sensor system comprising a 2-dimensional array of a photo-sensors for acquiring an image on a focal plane:
1. Shuttering. This approach uses externally controlled, variable integration times to attempt to increase the dynamic range of a sensor system. The variable integration time can be implemented electronically or manually, but in any case, this method fails when a scene includes a wide range of intensities. For example, if the shuttering control system shortens the integration time to avoid saturating the pixels, the dim part of the image is buried under the noise. If the shuttering mechanism lengthens the integration time to avoid cutting-off the weaker pixels, some pixels will saturate thereby causing the brighter portions of the image to be lost. Although this method expands the overall dynamic range of the sensor system, the dynamic range at any particular setting remains unchanged at Ss/Smin.
2. Local Shuttering. This approach determines integration times for small, local sub-regions within a sensor array. Typically, a plurality of integration control circuits are located within the focal plane of the sensor array and each of them controls the integration time of a block of pixels. The method requires substantial area overhead and computation to piece together the final image. Although all the pixels in a local sub-region have the same integration time, this method does widen dynamic range of the overall sensor system. Assuming the minimum integration time is T and the maximum integration time is kT, where k greater than 1, then the minimum detectable intensity is proportional to Smin/kT and the maximum non-saturating, detectable intensity is proportional to Ss/T. Thus, the overall dynamic range is kSs/Smin, a factor of k greater than the original dynamic range. However, individual pixels within a local sub-region can still saturate or be cut-off.
3. Individual Pixel Reset. This scheme uses separate external exposure control for each individual pixel. Like local shuttering, this method requires feedback, but it requires an even greater amount of memory and processing to set the integration time for each pixel. Like local shuttering, this method also increases dynamic range by a factor of k.
4. Local Gain Control. One prior art attempt at increasing a sensor""s effective dynamic range by gain control is described in U.S. Pat. No. 5,614,948 by Hannah. This approach determines an optimized gain for each analog processing circuit corresponding to a small, local sub-region comprising several pixels within a sensor array. This method also requires feedback, substantial memory, and a substantial amount of processing to set the gain of each sub-region. Again, like local shuttering and individual pixel reset, this method also increases dynamic range by a factor of k.
5. Floating-point ADCs. This method attempts to increase the overall dynamic range of a sensor system by directly performing floating-point analog-to-digital conversion of the analog signal. This technique can be understood with reference to FIG. 3 which shows a simplified block diagram of a prior art floating-point ADC system 50. Floating-point ADC system 50 comprises a programmable gain amplifier (PGA) 52, a fixed-point ADC 54, and a range selection circuit 56. Range selection circuit 56 provides an M-bit digital feedback signal that simultaneously serves as the exponent for the floating-point conversion and as the signal that sets the gain of PGA 52. PGA 52 receives an analog input signal V(t) and amplifies it with a gain corresponding to the digital code from range selecting circuit 52 to produce analog signal S(t). Fixed-point ADC 54 receives this amplified analog signal S(t) from PGA 52 and converts it to an N-bit digital representation that serves as the mantissa for the floating-point conversion. For each analog-to-digital conversion, range selecting circuit 56 must step through several digital codes (exponent values) until it sets the gain of PGA 52 such that the N-bit code from fixed-point ADC 54 lies somewhere between the minimum and maximum digital codes of fixed-point ADC 54. Because of the need for PGA 52 and range selecting circuit 56, this techniques requires extra circuit area. This technique also requires a relatively long conversion time to perform the iterative conversion process. Although this method increases the dynamic range of the digital output from the ADC relative to that of an N-bit fixed-point ADC, because the PGA equally amplifies both signal and noise, this method does not increase the fundamental dynamic range of the sensor system. The system""s dynamic range is still limited to Ss/Smin.
6. Temporary Capacity Saturation. This method attempts to increase the dynamic range of a sensor system by compressing its response curve in a non-linear fashion as follows: From time t=0 to time t=Td, the system is altered such that the sensor can integrate charge only up to a faction, xcex2, of its normal capacity (O less than xcex2 less than 1). Any excess charge beyond xcex2Qmax that is produced between t=0 and t=Td is drained off. At t=Td, the sensor""s capacity is restored to its full value, and the system then continues to integrate until a later time t=T. FIG. 4 pictorially illustrates this technique. FIG. 4 plots collected charge versus time for two curves corresponding to different lighting intensities. Curve A corresponds to a strong lighting condition while Curve B corresponds to a weak lighting condition. Note that until time Td, whenever the integrating photo-current I exceeds xcex2Qmax/Td, the collected charge is clipped at xcex2Qmax. The collected charge as a function of input photo-current I is plotted in FIG. 5 and can be expressed as:       Q    ⁡          (      T      )        =      {                                                      xe2x80x83                        ⁢                          I              ⁢                              xe2x80x83                            ⁢              T                                                                          xe2x80x83                        ⁢                                          if                ⁢                                  xe2x80x83                                ⁢                0                            ≤              I              ≤                              β                ⁢                                  xe2x80x83                                ⁢                                                      Q                    max                                    /                                      T                    d                                                                                                                                      xe2x80x83                        ⁢                                          β                ⁢                                  xe2x80x83                                ⁢                                  Q                  max                                            +                              I                ⁡                                  (                                      T                    -                                          T                      d                                                        )                                                                                                        xe2x80x83                        ⁢                                          if                ⁢                                  xe2x80x83                                ⁢                β                ⁢                                  xe2x80x83                                ⁢                                                      Q                    max                                    /                                      T                    d                                                              ≤              I              ≤                                                                    Q                    max                                    ⁡                                      (                                          1                      -                      β                                        )                                                  /                                  (                                      T                    -                                          T                      d                                                        )                                                                                                                    xe2x80x83                        ⁢                          Q              max                                                                          xe2x80x83                        ⁢                                          if                ⁢                                  xe2x80x83                                ⁢                I                             greater than                                                                     Q                    max                                    ⁡                                      (                                          1                      -                      β                                        )                                                  /                                                      (                                          T                      -                                              T                        d                                                              )                                    .                                                                        
Assuming Qmax corresponds to Ss, to compute the dynamic range of this method, let Imin=Qmin/T and Imax=Qmax(1xe2x88x92xcex2)/(Txe2x88x92Td). The dynamic range is therefore Imax/Imin=[Qmax/Omin](1xe2x88x92xcex2)/(1xe2x88x92Td/T). Thus, the dynamic range is increased by a factor of (1xe2x88x92xcex2)/(1xe2x88x92Td/T). For example, if xcex2=xc2xc and Td=3T/4, the dynamic range is increased by a factor of 3. This technique is a very crude approximation to a general logarithmic compression modeled by the following equation:
Q((T)=[Qmax/k)]log(1+I/Io) 
where Q is the collected charge and I is the input photo-current. From this equation, we have Imin=Io[e(kQ min/Q max)xe2x88x921] and Imax=Io[ekxe2x88x921] giving:
Dynamic Range=[ekxe2x88x921]/[e(kQ min/Q max)xe2x88x921]≈[(ekxe2x88x921)/k](Qmax/Qmin) 
if Qmax/Qmin greater than  greater than 1
Thus, this general method of logarithmic widening of the dynamic range results in a gain of (ekxe2x88x921)/k. Although this is a respectable increase in dynamic range, implementing this method can be very complicated and the pixels tend to be non-uniform.
Clearly, all these prior art attempts at increasing the dynamic range of an integrating sensor system suffer from either inadequate performance or complicated and expensive implementation (i.e. circuitry), or both.
One other prior art technique that attempts to extend the dynamic range of a sensor system in a very different manner from those discussed above was presented at the 1997 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors. In the paper entitled xe2x80x9cRecent Progress of CMD Imagingxe2x80x9d by Tsutomu Nakamura and Kuniaki Saitoh, a technique is described wherein an image sensor array is scanned twice. The first scanning outputs signals with a short exposure time, and the second scanning outputs signals with a long exposure time. The two signals are then synthesized to form an image with greater dynamic range than that which is possible with only one scanning. Although this prior art technique can improve the dynamic range of the system by scanning twice and outputting signals after two different integration times, it does not provide a means for converting the signal to digital form. Complex external circuitry is needed to combine the signals and perform analog-to-digital conversion. It is not a technique for performing analog-to-digital conversion.
It is therefore an object of the present invention to provide a method and apparatus for extending the dynamic range of an integrating sensor system that overcomes the limitations of the prior art. In particular, it is an object of the present invention to provide a method and apparatus that effectively extends the dynamic range of an integrating sensor system while requiring relatively simple circuitry and small circuit area.
It is also an object of the present invention to provide a method and apparatus that effectively extends the dynamic range of an integrating sensor system and provides a binary, floating-point representation of the sensor system""s analog signal.
It is a still further object of the present invention to provide a method and apparatus that effectively extends the dynamic range of an integrating sensor system while producing a bit-serial output that conforms to pixel level analog-to-digital conversion as described in U.S. Pat. No. 5,461,425 by Fowler et. al.
Accordingly, using an analog-to-digital converter with a maximum input signal of Ss, the present invention provides a method and apparatus for converting a monotonically changing analog signal to a cumulative floating-point, digital representation even if the analog signal has a value greater than Ss at time t=T. The method comprises the following steps: First, the analog signal is reset to a reference value at an initial time t=0. Then, the analog signal is sub-converted at a first time T1 greater than 0 to obtain an first digital representation which corresponds to the magnitude of the analog signal at this first time. Next, the analog signal is sub-converted at a subsequent time T2 greater than T1 to obtain a second digital representation which corresponds to the magnitude of the analog signal at this second time. The two digital representations are then combined into an intermediate floating-point, digital representation with greater dynamic range than either of the first two digital representations on their own. The above steps of sub-converting the analog signal and combining each new digital representation with the existing intermediate digital representation to obtain a new intermediate representation are repeated for other subsequent times T3 greater than T2, T4 greater than T3, . . . TM greater than TMxe2x88x921, in order to produce a cumulative floating-point, digital representation of the analog signal at time t=T.
An embodiment of an apparatus for converting a monotonically changing analog signal to a cumulative floating-point, digital representation with a wide dynamic range according to the present invention comprises the following: a) a means for resetting the analog signal to a reference value at an initial time t=0; b) a first signal generator for generating a first signal having a plurality of levels; c) a comparator having a first input connected to receive the first signal and a second input connected to receive the analog signal; d) a binary signal generator for generating a series of binary signals; and e) a latch having a first input coupled to receive the output signal from the comparator. The latch also has a data input that receives the binary signal. The output signal from the comparator controls when the latch provides an output signal that corresponding to the binary signals.
The circuit provides a first N-bit digital code representing the analog signal at a first sub-conversion time T1 greater than 0. The circuit also provides at least a portion of subsequent N-bit digital codes representing the analog signal at subsequent sub-conversion times T2 greater than T1, T3 greater than T2, . . . TM greater than TMxe2x88x921. A combination of the first N-bit digital code and the portions of the subsequent N-bit digital codes produces a wide dynamic range, floating-point, digital representation of the analog signal at time t=T.